Access latency to main memory (external memory) has lagged behind increases in processor speed resulting in a performance bottleneck. To decrease access latency, many processors include integrated on-chip caches that hold large contiguous data blocks (cache lines) fetched from main memory based on spatial and temporal locality. Spatial locality is the concept that the likelihood of referencing data is higher if data near it was just referenced. Temporal locality is the likelihood that data that is referenced at one point in time will likely be referenced again sometime in the near future.
Though many applications have data access patterns that exhibit temporal and spatial locality, there also exist classes of applications with data access patterns that do not. For example, some multi-media applications, database and signal processing applications do not exhibit a high degree of temporal and spatial locality. Also, some stride access patterns and indirect access patterns used in many data intensive applications do not exhibit a high degree of temporal and spatial locality.
Off-chip communication in conventional cache architectures is inefficient because data management is sized by cache line. If data access patterns do not exhibit spatial locality, only a small portion of a cache line is actually used, and memory bandwidth used for accessing the other part of the cache line is wasted. In addition, because data buffering is also based on an entire cache line, the efficiency of cache is low, resulting in more cache misses and more off-chip communication.
In addition, conventional processor architectures do not take advantage of parallelism in memory accesses. To prepare operands of a computation, that is, values on which an instruction operates, a processor may incur large overheads such as address calculation and data format conversion in addition to the actual memory accesses. Though pure memory latency is one cause of the performance bottleneck, the memory access overhead also contributes to access latency.
Although the following Detailed Description will proceed with reference being made to illustrative embodiments of the claimed subject matter, many alternatives, modifications, and variations thereof will be apparent to those skilled in the art. Accordingly, it is intended that the claimed subject matter be viewed broadly, and be defined only as set forth in the accompanying claims.